Skip to content

[AUTHZ] Authz shaded include the transitive jackson libs#5496

Closed
yikf wants to merge 4 commits intoapache:masterfrom
yikf:jackson
Closed

[AUTHZ] Authz shaded include the transitive jackson libs#5496
yikf wants to merge 4 commits intoapache:masterfrom
yikf:jackson

Conversation

@yikf
Copy link
Contributor

@yikf yikf commented Oct 23, 2023 •
edited
Loading

Why are the changes needed?

As description #5427 (comment). This PR aims to:

  • Authz shaded include the transitive jackson libs
  • Add LICENSE and NOTICE file to exactly match the content of jar
  • Add scala-2.13 profile to make authz shade module support scala 2.13

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

Was this patch authored or co-authored using generative AI tooling?

No

@yikf
Copy link
Contributor Author

yikf commented Oct 23, 2023

@pan3793

@pan3793
Copy link
Member

pan3793 commented Oct 23, 2023 

.
├── META-INF
│   ├── AL2.0
│   ├── ASL2.0
│   ├── LGPL2.1
│   ├── LICENSE
│   ├── MANIFEST.MF
│   ├── NOTICE
│   ├── jersey-module-version
│   ├── persistence.xml
...

I did a quick check on the shaded jar, and found the following issues:

  • where does LGPL2.1 come from? LGPL2.1 is under category-x which is not allowed to be included into ASF artifacts.
  • AL2.0, ASL2.0 should be excluded
  • LICENSE and NOTICE should be modified to exactly match the content of jar

@codecov-commenter
Copy link

codecov-commenter commented Oct 23, 2023 •
edited
Loading

Codecov Report

Merging #5496 (758d834) into master (03d6223) will not change coverage.
Report is 16 commits behind head on master.
The diff coverage is n/a.

❗ Current head 758d834 differs from pull request most recent head 8eeec2b. Consider uploading reports for the commit 8eeec2b to get more accurate results

@@          Coverage Diff           @@
##           master   #5496   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         588     588           
  Lines       33480   33480           
  Branches     4405    4405           
======================================
  Misses      33480   33480

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pan3793
Copy link
Member

pan3793 commented Oct 23, 2023

BTW, jna seems to be the dep of scala 2.13

@yikf
Copy link
Contributor Author

yikf commented Oct 23, 2023

  • where does LGPL2.1 come from? LGPL2.1 is under category-x which is not allowed to be included into ASF artifacts.

LGPL2.1 comes from net.java.dev.jna, and it is licensed under the Apache License, excluded LGPL2.1 from shaded jar looks like fine.

BTW, jna seems to be the dep of scala 2.13

jna looks like come from ranger

@pan3793
Copy link
Member

pan3793 commented Oct 23, 2023

BTW, jna seems to be the dep of scala 2.13

I mean apache/spark#43008 (comment)

@yikf
Copy link
Contributor Author

yikf commented Oct 23, 2023

I mean apache/spark#43008 (comment)

Thanks for the information. It looks like we need to provide the scala-2.13 profile, which requires exclude jna.

For the scala 2.13 profile, maybe the jna version of ranger is different from the jna version of scala 2.13, but I can't think of a good way to solve it, because jna cannot shade.

@pan3793
Copy link
Member

pan3793 commented Oct 23, 2023

@yikf I suppose we can use the jna provided in Spark, then let's excluding jna in scala-2.13 profile

@yikf yikf changed the title [AUTHZ] Autz shaded include the transitive jackson libs [AUTHZ] Authz shaded include the transitive jackson libs Oct 23, 2023
@yikf yikf self-assigned this Oct 23, 2023
@yikf yikf added this to the v1.9.0 milestone Oct 23, 2023
pan3793
pan3793 reviewed Oct 24, 2023
View reviewed changes
extensions/spark/kyuubi-spark-authz-shaded/pom.xml
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<artifactSet>
<excludes>
Copy link
Member

@pan3793 pan3793 Oct 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have you tested it? what's the behavior when both includes and excludes are present?

Copy link
Contributor Author

@yikf yikf Oct 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will contain the content in the includes tag and not in the excludes tag.
image

@pan3793 pan3793 closed this in 322cb48 Oct 27, 2023
@pan3793
Copy link
Member

pan3793 commented Oct 27, 2023

Thanks, merged to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pan3793 pan3793 pan3793 approved these changes

Assignees

@yikf yikf

Labels

kind:build module:extensions module:spark

Projects

None yet

Milestone

v1.9.0

Development

Successfully merging this pull request may close these issues.

3 participants

@yikf @pan3793 @codecov-commenter